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[57] ABSTRACT 

A method and apparatus for splicing a first compressed 
digital information stream into a second compressed digital 
information stream. The first information stream includes at 
least one entrance indicium identifying an appropriate 
stream entrance point, the second information stream 
includes at least one exit indicium identifying an appropriate 
stream exit point. A controller monitors the two streams until 
the appropriate points are found and, in response to a control 
signal, splices the first stream into the second stream. 

13 Claims, 5 Drawing Sheets 
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METHOD AND APPARATUS FOR SPLICING stream. The invention monitors the two streams until the 

COMPRESSED INFORMATION STREAMS appropriate points are found and, in response to a control 

signal, splices the first stream into the second stream. 

This application claims the benefit of U.S. Provisional Specifically, the inventive splicer includes a pre-splice 

Application No. 60/018,554, filed May 29, 1996. 5 buffer receiving a first information stream and producing a 

This application is related to U.S. patent application Ser. buffered information stream; a bitstream examiner receiving 

No. 08/864,321 (Attorney Docket No. 12070), filed on the the first information stream and rcsponsively causing the 

same date as the present application. pre-splice buffer to position an entrance point of the buffered 

This invention was made with U.S. government support information stream at an output of the buffer; a switch for 

under contract number 70NANB5H1174. The U.S. Govern- io coupling either the buffered information stream or a second 

ment has certain rights in this invention. information stream to an output; and a switch controller for 

The invention relates to communication systems in monitoring the second information stream and, in response 

general, and more particularly, the invention relates to a to a control signal and the detection of an exit point in the 

method and apparatus for synchronizing a plurality of com- second information stream, causing the switch to couple the 

pressed data streams to facilitate stream selection, splicing 15 buffered information stream to an output. 



and other operations. 

BACKGROUND OF THE DISCLOSURE 



BRIEF DESCRIPTION OF THE DRAWINGS 



The teachings of the present invention can be readily 

In several communications systems, the data to be trans- understood by considering the following detailed descrip- 

mitted is compressed so that the available bandwidth is used 20 tion in conjunction with the accompanying drawings, in 

more efficiently. For example, the Moving Pictures Experts which: 

Group (MPEG) has promulgated several standards relating piG. 1 shows a block diagram of a compressed bitstream 

to digital data delivery systems. The first, known as MPEG-1 splicing system including the invention; 

refers to ISO/IEC standards 11172, incorporated herein by l7Tr , - , . . , flrtw . „ . nf „ mmUmm ■ „ 

reference. The second, known as MPEG-2, refers to ISO/ 25 . F^. 2 depicts a flow chart of a seamless sp hang process 

IEC standards 13818, incorporated herein by reference. A m accordan ~ ™ th thc ™™ l ™> 

compressed digital video system is described in the FIG. 3 shows a detailed block diagram of the splicer of 

Advanced Television Systems Committee (ATSC) digital mG - 

television standard document A/53, incorporated herein by FIG. 4 depicts a block diagram of digital studio compris- 

reference. 30 ing a plurality of interoperable islands and including the 

A program transport stream is formed by multiplexing invention; and 
individual elementary streams which share a common time FIGS. 5A-5C depicts a plurality of splicing scenarios, 
base (i.e., the same 27 MHz clock source). The elementary To facilitate understanding, identical reference numerals 
streams comprise encoded video, audio or other bit streams. 35 have been used, where possible, to designate identical 
The elementary streams may be, but do not have to be, in a elements that are common to the figures, 
packetized elementary stream (PES) format prior to trans- 
port multiplexing. A PES consists of a packet header fol- DETAILED DESCRIPTION 
lowed by a packet payload. As the elementary streams are Toe invention is generally described within the context of 
multiplexed, they are formed into transport packets and a 4Q a digital television studio includes a plurality of operative 
control bit stream that describes the program (also formed environments which receive and process various bitstreams 
into transport packets) is added. anc j which have associated switching capabilities according 

There are many instances where there is a need to switch to the invention. The switching capabilities allow seamless 
from one encoded or compressed bitstream to another. When or non-seamless splicing of a plurality of, e.g., video trans- 
switching from one compressed MPEG video bitstream to 45 port streams to produce an output stream. A combination of 
another, appropriate measures must be taken in the trans- seamless and non-seamless bitstreams may be produced to 
mission order of the picture bitstream to assure proper provide a controllably degraded output stream, 
subsequent presentation of the decoded pictures, without The invention is a two-input bitstream splicer which 
time gaps. Such time gaps result in undesirable video or performs switching, splicing or insertion operations on a 
audio artifacts (e.g., blank screen due to buffer overflow/ 50 pair of MPEG -compliant input transport streams to produce 
underflow, poor "Up sync" and the like). Heretofore there an output stream. It must be noted that the principles of the 
has not existed a seamless splicing method and apparatus for invention apply to bitstream switchers or splicers having 
splicing transport streams to one another. m0 re than two inputs and to input streams other than 

Therefore, a need exists in the art for a method and MPEG-compliant input streams. The invention may be 

apparatus for splicing compressed digital informadon bit- 55 implemented using a general purpose computer system that 

streams. is programmed to perform the functions discussed below. As 

„, „ . „„, ^ , programmed, the general purpose computer becomes a spe- 

SUMMARY OF THE INVENTION cific pmpose apparams ^ sp ^ cing ^ ^ bU strea £. 

The disadvantages heretofore associated with the prior art The invention may be used for both seamless and non- 
are overcome by the present invention of a method and 60 seamless splicing of bitstreams. Seamless splicing means 
apparatus for splicing compressed digital information seamless butt-splicing of two streams to form a resultant 
streams. In particular, the invention splices a first informa- output stream that produces a continuous, undisturbed flow 
tion stream into a second information stream. The first of information (e.g., video or audio without glitches or 
information stream includes at least one entrance indicium artifacts). Non-seamless splicing produces a resultant output 
that identifies an appropriate point of entrance to the stream. 55 signal which may have a disturbed information flow (e.g., 
The second information stream includes at least one exit visual or aural distortions, disturbances and artifacts). For 
indicium that identifies an appropriate point of exit from the purposes of this discussion, it will be assumed that each 
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bitstream is a transport stream comprising video, audio and One specific example of a valid splice segment that can be 

(possibly) other information. It must be noted that the seamlessly spliced is an MPEG-compliant splice segment, 

invention is applicable to packetized elementary and other An MPEG Splice Segment (SS) is defined at the transport 

elementary streams. Additionally, it is assumed that the level and includes functionality at the video (and audio) 

splicing points are determined with respect to the video 5 levels. An information-bearing splice segment may be as 

information. This may result in some distortions in the short as a single frame. A splice segment may even be azero 

spliced audio and other information, since the audio and & ame ^ent (although such a SS might be MPEG 

other information may not temporally "hue up" on a packet non-compliant) Such a zero-lengtb segment is simply an 

by packet basis in-point followed by an out-point (i.e., an ln-out-poinf ). A 

* . '. t . . . ... . . lfl SS may be also be very long, including many GOPs. In 

Splicing consists of making a transition in an output- { ^ { ± of a 55 ^ not conslramed and the SS 

stream from a from-stream to a to-stream The from- should out mts to enable X9mkss exit . 

stream is ideally exited at an ou -point and the to-stream from ^ £ A ^ {qd ^ & ss CQm . 

is ideally entered at an "in-point. An out-point is a place in * a televis f on comme F rciah ^ television 

a presently-selected stream (i.e., "from-stream ■) where the ^ ^ be deliberately produced without defined out-points 

stream may be ended, and some other stream (ie., to- M ^ commerdal m ^ not seamless . 

stream ) sphced on. An "in-point is a place m the other WI1I _ to oc , .... ,. , t 

stream where the information may begin to be sphced on to fc MPEG , SS ( f hould * e » ? om P h f ant s f eam 

, /or havmg consistent transport stream and elementary stream 

anotner stream. ^ ^ pcR ^ ^ DTS) ^ an 3^^^ 

A "splicing segment" is defined as the portion of an dd parameter ( e .g. ? a yBV delay), thereby allowing a 

information stream between an in-point and an out-pomt. A decoder tQ properly decodc and prcsent me information m 

splicing segment may include multiple out-points and mc 55 ^ first information framc (e . g . } video acccss unit) 

in-points. Thus, it is desirable to include as many in-points at m m int of aQ MPEG vidw ss must bc aQ T . Framc ^ 

and out-points as possible in a stream to allow for maximum secoa(J frame &Ml QQt reference mformation framcs prior t0 

flexibility in splicing. Within the definition of an in-point (hc . int ( . if thc sccond frame is a B . fram6) thc 

and an out-pomt is a delay-parameter, e.g., a video buffering B . frame M rcference frames ior tQ the ^ ^ ^ 

verifier (VBV) for MPEG comphant streams. A splicing last frame before an out . point should not be a B . frame (in 

segment with a known in-point delay-parameter and with di& , Qrder) ^ audio ss ^ haye fln in ^ consisling 

out-points having the same known delay-parameter may of ^ beginning of ^ audio frame ^ m out . point 

mclude within itself shorter valid splicing segments with sistingofthe last byte of an audio frame. There may be other 

different values of the delay-parameter. constraints placed on the stream to address issues of, e.g., 

In the context of a studio environment of an exemplary coding error-build-up, tuning-time and minimum picture 

embodiment of the invention, information streams are quality. 

divided into transport packets. Packets containing video may ^ ^.po^ 0 f a video 55 mus t begin with a sequence 

be intermixed with packets containing audio, auxiliary data, 35 header, although the SS may contain multiple sequence 

or other information. In this environment, a video stream headers. ASS may contain additional header information to 

out-point is the end of the last video transport packet of the indicate that the sequence header is also an in-point. It is 

stream of interest. The video stream before and through the necessary to distinguish the SS in-point sequence header 

last packet must meet the splicing definition of an out-point. from a sequence header included for tuning-time or picture 

Similarly, a video stream in-point is the beginning of the first ^ qua hty s in Ce seamless splicing can only be guaranteed on 

video transport packet of a splice segment (SS). It must be i n .points. Since the in-point should follow an sequence end 

noted that other information in the transport stream, specifi- code ( SE q code it ^ desirable to include the SEC code just 

cally audio, is unlikely to be neatly segmented at in-points before the ^by obviating the need to include the 

and out-points. A method for correcting errors induced by SEC on me end of an ou^im. The out-point may include 

the non-alignment of audio transport packets is described in 45 the S£C ^ M p EG -type splice count-down, if used, must 

U.S. patent application Ser. No. 08/864,321, filed simulta- end (j c cqua , zero ) at tQe oul . pomt . 

neously herewith (Attorney Docket 12070), incorporated FIG. 1 shows a block diagram of a compressed bitstream 

herein by reference. splicing system 100 including the invention. The system 100 

A critical aspect of splicing information streams is the includes a first compressed bitstream stream source 110, a 
proper processing of the various delay parameters. One 50 second compressed bitstream stream source 120, a splicer 
parameter of concern is the delay parameter associated with 300, a controller 105 and an optional splice monitor 130. 
the various information streams. In the case of an MPEG- The first compressed bitstream stream source 110, illustra- 
compliant stream, the delay parameter is the video buffering ^ wc \ y a «h ve feed" from a transport stream encoder, pro- 
verifier (VBV) delay parameter. Another parameter is the duces a first MPEG-compliant transport stream S6. The 
latency, or transitional period, inherent in a splicing opera- 55 second compressed bitstream stream source 120, illustra- 
tion. For example, a typical splice occurs at a certain time, t j ve ] y a server (e.g., a video disk, tape machine, or other 
i.e., a "splice time." Prior to the splice time an output storage device) which stores video and audio elementary 
information stream comprises a from-stream. At the splice streams and transport encodes the stored streams to produce 
time, a switch to the to-stream occurs. For a period of time a second MPEG-compliant transport stream S7. The stored 
the output stream may include information from both the 60 information may comprise, e.g., advertisement or local 
from-stream and the to-stream. Eventually the output stream programming information to be sphced into the first trans- 
includes information from only the to-stream. port stream. The splicer 300 selectively couples one of the 

It is assumed that the from-stream and the to-stream are two input transport streams S6, S7 to a transmitter or other 

each valid. There are certain constraints on the streams that subsystem as an output stream S9. An optional splice 

must be met if the splicing is to be seamless. Seamless 65 monitor 130 monitors various parameters of the spliced 

splicing implies that the resultant spliced bitstream will not output signal S9, e.g., delay parameter, buffer utilization 

cause discontinuities in the future. information, synchronization, bitstream source and the like. 
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The optional splice monitor 130 is responsive to the con- The timing relationships between desired operation and 

troller 105 and the splicer 300. actual operation will be discussed first. At some operational 

The splicer 300 receives the first transport stream S6, unit . e -g- a Play-to- Air Switcher, a decision to switch 
illustratively a television program produced by a first source, streams must be made. The source of an output stream is 
and the second transport stream S7, illustratively an adver- 5 actually switched in response to that decision, 
tisemenl produced by a second source. In response to a The decision to splice may be content related, such as a 
control signal SELECT, the splicer produces an output switch from a from-stream to a to-strcam when a content- 
signal S9 comprising either the first S6 or second S7 related data element is encountered in one of the streams, 
transport stream. The control signal SELECT may include For example, the from-stream may be monitored and, in 
priority information which causes the splicer 300 to respond 10 response to the detection of, e.g., a black-screen or a scene 
immediately, within a defined time interval or when certain change, a splice decision may be made. This operational 
conditions exist (i.e., specific alignments of stream entrance decision does not require synchronization. Rather, the deci- 
or exit points). The splicer 300 produces a signal sion requires that the splicer (or a controller) analyze, e.g., 
ACKNOWLEDGE which is used to acknowledge the the from-stream to detect the data element. The decision to 
SELECT signal and provide specific details about the splice 15 splice may also be data-flow related, such as a switch from 
operation (e.g., exact time of splice, error conditions and the a from-stream to a to-stream on some particular packet or 
like). The operation of the splicer 300 is described more upon the start or stop of information flow, 
fully below with respect to FIG. 3. The decision to splice may be time-related, such as a 

The actual splicing operation is the process that takes switch from a program to commercial at noon. Time-related 
place within the splicer 300 that does what is necessary to 20 decisions must be referenced to the splicer's local frame- 
actually switch amongst the bitstreams. This involves of-reference. A message-passing process passes the decision 
stopping, in an orderly manner, the flow of packets from the information to the splicer in time for the splicer to be ready 
from-stream; starting, in an orderly manner, the flow of to make the splice in its frame-of-reference. Given that the 
packets from the to-stream; and adjusting the header infor- decision to splice at some time has been made, the splice will 
mation in the output stream. During some interval, packets 25 be made at the next available splice point, based upon the 
from both the from-stream and the to-stream are likely to be from-stream and the to-stream. 

intermixed. The decision to splice may be may be event driven, such 

Splicing operations must be synchronized to be seamless. as the pushing of a button (e.g., the director's "take" 

To ensure that input streams arrive at the appropriate splicers 3Q command, as depicted in the splicer 100 of FIG. 1). When 

at the time they are needed several synchronizing operations the message indicating the event arrives at the splicer, the 

may be performed. It is assumed that the output stream is actions are the same as those for a time-related decision 

continuous and that the actual splice is taken to be a change whose time has arrived. 

in the content of the output stream from a from-stream to a Some form of acknowledge message may be required, 

to-stream. The time stamps in the output stream should also 35 This message, when delivered to the originator of the splice 

maintain continuity from one stamp to the next (this is decision (e.g., the controller), will allow an intelligent 

related to stream content) and the splicing mechanism choice to be made about time-outs, and actions like panic 

should adjust the output stream time-stamps. In the absence non-seamless splices. Time-outs and determinations about 

of time stamp continuity in an MPEG system, the MPEG corrective actions to remedy splice failures is a policy matter 

"discontinuity" header flag should utilized such that an 4Q for the originator of the splice decision. Time-out and forced 

indication of a new time stamps (or time stamp switch may be a service implemented by the splicer but only 

discontinuity) is provided to a decoder. as a convenience. 

To accomplish the adjustment the splicing process must An operational unit (e.g., splicer or switcher) may feed 
have some notion of time, since this local notion of time that back an appropriate acknowledgment message to a control- 
must be used to produce the output time-stamps. The splic- 45 ling entity. The contents of such a feedback message may 
ing process gets its notion of time from some timing source include one or more of the following parameters: 1) a splice 
such as the OC-12c interface and the current time is derived did or did not take place; 2) the local time-of-day that the 
from either stream content or set-time messages. The local splice occurred; 3) the delay-parameter value of the 
notion of time must be moderately continuous and well to-stream; 4) the delay-parameter value of the from-stream; 
behaved. When splicing, both the end of the from-stream 50 5) the current (post-splice) sync-buffer delay (e.g., in delay 
and the beginning of the to-strcam must be available at the seconds); 6) the future time a splice will take place (if the 
actual splice hardware that is producing the output. In switcher can compute this value); and 7) any exceptions or 
addition, all buffering within the splicing process must be errors. Exceptions and errors may include the fact that no 
finite and defined. splice took place, that the decision parameters passed by the 

In addition to the above issues, there are synchronization 55 controller were incorrect (e.g., syntactically or logically), 

issues to be considered. For example, it is important to that the to-stream was not ready, that a time-out occurred or 

consider the effect of packet jitter on the splicing process. If that an audio-failure occurred (e.g., the dropping of an 

any additional information is required, beyond that con- excessive number of audio frames), 

tained within the actual streams being spliced (e.g. priority Additional information that may be of value includes: 1) 

information, source identification, error codes and the like), go the amount of time that the audio information from the 

the additional information must be properly synchronized from-stream will be needed; 2) an indication that the inputs 

with the actual splice streams. are buffered correctly and ready for a new splice; and 3) 

There are several conditions that are of interest with other information useful to the controller or the splicing 

respect to synchronization of the splicing function. These are process itself. 

the timing relationships between desired operation and 65 The precise time at which a seamless splice takes place 

actual operation, continuous-flow streams, server-generated may not be pre-determined, since the seamless splice 

streams and remotely-generated streams. depends upon the arrival of an in-point in the to-stream. In 
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the case of a decision to splice seamlessly there are several toring may comprise the steps of viewing an image on a 
sub-decisions which must be made about what to do if the display device (i.e., "monitor") responsively changing 
splice does not take place within some time limit. The parameters of the bitstreams producing the image (e.g., 
choices are as follows. First, simply wait for a seamless splicing). Optional splice monitor 130 may be used for 
splice to occur. Depending upon studio operational goals, 5 content-related monitoring by, e.g., a director. Another form 
this may not be acceptable. Second, define a fixed time-out of monitoring is the qualitative assessment of a monitored 
period and, if the splicer has not spliced within the defined bitstream. Optional splice monitor 130 may be used to 
time-out period, perform a non-seamless splice (i.e., switch retrieve qualitative information from the spliced output 
streams in as controllable a manner as possible). While this signal S9, e.g., delay parameter, buffer utilization 
is a simple approach, it limits decision flexibility. Third, information, synchronization information, bitstream source 
simply wait for the splice to occur but acknowledge this identification and the like. The optional splice monitor 130 
condition to the controlling entity. Whatever process gener- is responsive to the controller 105 and the splicer 300 to 
ated the splice decision may then decide to allow a non- either process the information and return, e.g., an opera- 
seamless splice. This allows reasonable operational flexibil- tional summary, or to couple the qualitative information 
ity. Fourth, provide a programmable time-out, optionally directly to the controller 105 and the splicer 300 for further 
including a default time-out and optionally including an 15 processing 

acknowledgment of this condition to the controlling entity witfa respect tQ monitoring> a director (i . e . s 

This option allows the controlling entity to alter the Ume-out hyjm&n) be y ^ vadous yfa 

value, thereby providing the greatest operational nexibmty. ,. ?■ , \ , -, ,-• 

It should be noted that a*time*ut is a secondary operational ^ decoders driving displays) and responsive^sphcing to 

feature and, therefore, an exception to normal operation. 20 P roduc * an 0U lP ut stream ( ie ". a P"?"")' This issue is 

The synchronization of continuous-flow stream splicing especiaUy significant m production or live switching with 

will now be discussed. The streams arrive and must be ^F^^«g^i»«iirt 

buffered (e.g., one second's worth of information) within the rf operating in a low-delay (e.g., I-frame only) production 

splicer in some manner so that, e.g., to-stream in-points are mode - ,he Sector indicates that a splice is to be made 

available when out-points occur in the from-stream. If the 2 5 " now >" that decision is likely to be based upon what the 

amount buffered is insufficient (e.g., more than a second director sees and expects to see. As a decoder inherendy 

elapses between successive in-points in a from-stream), then presents its picture after some delay from its input bitstream 

the buffer will overflow and will contain invalid information. ( e -g > P art or a ^ of end-to-end delay and some additional 

This condition is remedied by an appropriate number of decode and format delay), the modes of operation for the 

in-points and out-points being inserted into the bitstreams. If 30 director in an MPEG studio environment will be different 

bitstreams do not have in-points and out-points often from current NTSC-related operations. For purposes of the 

enough, then those bitstreams can not be seamlessly spliced following examples, assume that a director is operating in an 

at those times. Moreover, to the extent that there is packet or MPEG studio environment, that splice segments are Vi 

cell jitter in the arrival time of input bitstreams, a first-in, second long, and that the end-to-end delay of the spliced 

first-out (FIFO) buffer (with output clocked at nominal data 35 stream is V2 second. 

rate) is expected to smooth the flow, A first example is a "soonest" mode of operation. In this 

The synchronization of server generated streams will now mode, the director presses a "take" button TAKE based upon 

be discussed. Server-generated streams must be carefully an event seen on an output monitor 132, a from-stream 

generated so that the data does not arrive at the splicer too monitor 136 or a to-stream monitor 134. A queued up (e.g., 

early or too late. If the data arrives too early, there is some 40 server-stored) to-stream is ready and aligned at an in-point. 

risk of overflow of an input buffer. If it is assumed that the Within Vi second of the splice command an out-point will 

splicer has enough synchronization buffering to bold a arrive at end of the from-stream sync-buffer and the splice 

second or so of video, then it would seem that server streams will be made. It must be noted that the from-stream contains 

can be delivered in any pattern of flow that never exceeds the up to V* second of delay. One monitor delay later (V2 second) 

just-in-time limit, and the one-second-early limit. Of course, 45 the scene on the output monitor changes, 

there may be peak rate imitations on the splicer. If the director responded to a scene on the output monitor 

The synchronization of remotely-generated streams will 132, the amount of output monitor delay (i.e., the lime 

now be discussed. It must be noted that any stream pro- between the "take" command TAKE and a change in scene 

cessed in a studio containing the splicer is expected to have on the output monitor 132) is between Vi and one second. If 

the same reference clock rate. Remotely-generated streams, 50 the director responded to a scene on the from-stream moni- 

by the time they have reached a splicer, should be the same tor 136 , the amount of from-stream monitor delay is 

as locally-generated real-time streams. To reference between Vi and Vi second and the output monitor delay is VS 

remotely-generated streams to a local master clock the second. If the director responded to a scene on the to-stream 

remote source maybe genlocked to the local studio. This can monitor 134, the to-stream monitor 134 is continuous (i.e., 

be done via a reverse channel or by locking both to an 55 no monitor delay) and the output monitor delay is negative 

external reference, such as a timing signal derived form the V* seconds (Lc, the scene changes Vi second after the "take" 

Global Positioning System (GPS). If there are two indepen- button TAKE is pressed and the image displayed occurred Vi 

dent studios, each with an independent master clock, and second prior to the press of the button), 

each is doing a remote feed to the other, then one will be A second example is the "next" mode of operation. In this 

delivering data too slow for the other, and one will be eo mode, a queued up to-stream is flushed from a to-stream 

delivering data too fast. Another method is to delay a remote synchronization buffer and the next segment beginning with 

feed by a time equal to the maximum clock drift over some an in-point is queued up within up to Vi second. The 

operating interval. A 30 ppm. drift rate, over 24 hours, to-stream synchronization buffer also has zero to Vi seconds 

accumulates 2.6 seconds. A six second buffer, initialized to of random delay. When the in-point arrives the splice is 

a 3 second fill, is adequate to absorb clock drift. 55 made. 

Splice monitoring is an important aspect of splicing, If the director responded to a scene on the output monitor 

especially in a studio environment. Content-related moni- 132, the amount of output monitor delay is between VS and 
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one second. IE the director responded to a scene on the The delayed bitstream S3B is coupled to a working buffer 

from-stream monitor 136, the amount of from-stream moni- 330B and a switch controller 340. The second working 

tor delay is between Vi and V* second and the output monitor buffer 330B produces an output signal S4B which is coupled 

delay is ¥t second. If the director responded to a scene on the to packet switching unit 350. The second working buffer 

to-stream monitor 134, the to-stream monitor 134 is con- 5 330B holds the selected bitstream long enough to allow for 

tinuous and the output monitor 132 switches to a new scene overlap of old audio packets with current video packets. This 

between zero and V4 second later. aUow * ""Jo frames to continue to completion after a splice 

„ , . „„ „ ....... is made. The synchronization of audio and video frames are 

The choice of "soonest" or "next' mode of splicing is an discussed fa m ' ore detail below and in U.S. patent applica- 

! lp ^^^^, h ^T^. d T ! ^■ m ^on Scr. No. 08/864321, filed simultaneously herewith 

ing effect (delay or back-up) is least objectionable. To w (Atto Dockel 120 70) and incorporated herein by refer- 

alleviate these effects an amount of delay may be inserted ^ nce 

into the splicer inputs. If this delay matches the monitor A splice decision is made by a controller (e.g., controller 

delay, and the monitors are connected to the inputs of the ^ ^ (Q ^ switch controUer ^ * a a 

delays, then the apparent delay between monitor scenes and signal SELECT. Assuming that the splice decision equates to 

button action is less, but the delay to final output is greater. 15 * seamlessly at the next opportunity," 

In addition, a separate monitor control unit may be built to . . t f. , An i . X. \l 

simulate the bit-stream switching and show the simulated th f ™f ^T} r "T^ V SET 5 ! ^ I 

results of the bitstream switch, thereby providing more selected °ujP u j stream (i.e., bitstream S3B) for out-points. It 

flexibility to the director. 15 aa ? une . d that ™ in ' po ^ P° s ' tl0ncd at cnd of * c first 

y 2n synchronization buffer 320A. When an out-point arrives on 

FIG. 3. shows a detailed block diagram of the splicer 300 the from . stream , the switch controller 340 causes, via a 

of FIG. 1. The splicer 300 selects one of a first input cont[0 \ a/B, the switch 350 to begin coupling video 

bitstream 86 and a second input bitstream 87 as an output packets from the to-stream through the switch to an optional 

bitstream S8. The output bitstream 88 is optionally time header ad j uster At an appropriate time any audio packets 

stamped to produce a retimed output stream S9. The first and ^ t0 _ stream are also switched, 

second input bitstreams S6, 87 are, illustratively MPEG- ^ ^ ^ Q ^ time . stam to the 

compliant transport streams including at least video and selecte / output stream S8 to produce a retimed output 

audio elementary streams The video and audio elementary stfeam g9 ^ retimi of |he ^ 

streams may be in a packetized elementary stream (PES) (pcR)> pre560tation time 5tamps (PTS) and decode time 

tormat. ^ stamps (DTS) of the selected stream S8 may be necessary to 

For purposes of the following discussion it is assumed ensure that the splice is, in fact, seamless to a decoder. The 

that the second bitstream S7 is currently selected as the header adjuster 360 includes a 27 MHz (local) station clock 

output bitstream (i.e., S7 is the from-stream) and the first 362 which is utilized by a local PCR and PCRB generator 

bitstream S6 will be selected as the output bitstream (i.e., S6 364. To retime the presentation and decode time stamps it is 

is the to-stream) after a splicing operation. 35 necessary to partially decode (i.e., packetized elementary 

The first input bitstream S6 is coupled to a first bitstream stream (PES) layer) the selected transport stream S8. The 

examiner 310A and a first synchronization buffer 320A. The partial decoding and retiming of the PTS and DTS is 

first bitstream examiner 310A examines the first bitstream performed by a PTS and DTS detection and retiming unit 

for entrance points which have been included in the first 366 to produce a PTS and DTS retimed stream S8P. The PTS 

input bitstream S6. When an in-point is found, the contents 40 and DTS retimed stream is transport encoded and time 

of the synchronization buffer are discarded (i.e., the buffer is stamped by PCR detection and retiming unit 368 to produce 

"flushed") and the in-point is stored in the first memory a retimed transport stream S9. An alternate embodiment of 

portion of the synchronization buffer. The synchronization a header adjuster is the PTS-DTS retimer discussed in more 

buffer may be constructed as a first-in, first-out (FIFO) detail in U.S. patent application Ser. No. 08/864,326, filed 

buffer. The process of searching for in-points and flushing 45 simultaneously herewith (Attorney Docket 12389) and 

the buffer is repeated until the first input bitstream S6 is incorporated herein by reference. 

selected by the splicer. In this manner, there is always an As previously mentioned, the invention may be imple- 

in-point positioned at the end of the synchronization buffer mented using a general purpose computer system that is 

of the stream that is not being currently output. The output programmed to perform the various functions. The embodi- 

bitstream S3A of the first synchronization buffer 320A is 50 ment of FIG. 3 may be implemented as a computer program 

coupled to a switch controller 340 and a first working buffer utilizing portions of memory to provide buffering, and an 

330A. The first working buffer 330A produces an output algorithm directed to the examination, control, switching 

signal S4A which is coupled to a packet switching unit 350. an d header adjustment functions. As previously mentioned, 

The second input bitstream S7 is coupled to a second the splicer 300 produces a signal ACKNOWLEDGE which 

bitstream examiner 310B and a second synchronization 55 is used to acknowledge the SELECT signal and provide 

buffer 320B. If tbe second bitstream were not presently specific details about the splice operation (e.g., exact time of 

selected as the output stream, then the second bitstream splice, error conditions and the like). A routine for splicing 

examiner 310B and synchronization buffer 320B would will now be described with respect to FIG. 2. 

operate in the same manner as described above with respect FIG. 2 illustrates a splicing routine in accordance with the 

to the first bitstream examiner 310A and synchronization 60 invention. The splicing routine is entered at step 202 when 

buffer 320 A. The second bitstream examiner 3 llOAexamines the decision to splice is made. For the purpose of this 

the second bitstream for exit points which have been discussion, it is assumed that the decision is to seamlessly 

included in the second input bitstream S6. In the "selected splice from the currently selected (from) stream S4B to 

mode" of operation, the second bitstream examiner 310B is another (to-stream) stream S4A. The decision is examined at 

not used and the second synchronization buffer 320B serves 65 step 204. If the decision of step 202 is to splice as soon as 

as a constant delay buffer which produces a delayed bit- possible, then the routine proceeds to step 208. If the 

stream S3B. decision is to splice at the next in-point (e.g., skip the 
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presently buffered GOP in the to-stream), then the synchro- information streams from a NETWORK FEED and a 
nization buffer (e.g., 320A) is flushed. When the to-stream LOCAL FEED. Controller 105 communicates with each of 
synchronization buffer (e.g., 320A) has a valid in-point the islands via a control channel C. The control channel C is 
stored (step 208) and the from-stream is at a valid out-point used to direct the flow of information throughout the studio 
(step 210), the splice is made (step 220) and the routine is 5 (i.e., between islands) and to direct the processing of the 
exited (step 230). The context in which a splicing decision information within the islands. The controller 105 provides 
(step 202) is made is relevant to the amount of information the splicing decisions and any necessary parameters asso- 
necessary to perform a seamless splice. If the splice decision ciated with the intended splice. The islands respond by 
is made in the context of building play-to-air edit lists, it is performing, monitoring and acknowledging (via control 
necessary for the streams to be spliced to have the same J0 channel C) the various splicing operations, 
value of delay-parameter. If the splice decision is made in A digital studio according to the invention may be 
the context of creating a live production, it is necessary for described as an interconnected group of "splicing islands" 
the streams being spliced to have matching delay-parameters which perform particular processing functions on received 
and splice points which occur often enough to meet opera- bitstreams to produce output bitstreams. This is because the 
tional requirements. If the splice decision is made in the islands form individually distinct operating environments 
context of creating a live production, and the production 15 («*■■ st ? ra g« environments editing environments, process- 
involves information stored on servers, it is helpful to know m S environments and the like) which cooperate with each 
that anticipated splice-points are about to arrive soon (via, via * ^^^Si^i^"^^ 1 
eg., splice point countdowns or splicing tables). The splic- b t ltstreams S9 OUTPUT STREAM). Each is and oper- 
mgfccisioE , and related matters will be T discussed in detail M ate f s at a ^ ow J delay -parameter value and al splicing 
.7 20 within an island is (ideally) seamless. The splicing and 

° w * . .„ , , . , „ processing functions are under the general control of con- 

The invention will now be described within the context of troller 105> but may ^ locally contro i led as necessary. For 

a digital television studio including a number of distinct example, an operator sitting at an editing station may 

operating environments (such as servers or edit-suites) logically comprise one island. The streams to be edited are 

which receive, process and transmit various information 25 routed to an editing island (e.g., island 407) in response to 

streams. The operating environments, or "islands of commands transmitted via control channel C from controller 

interoperability," may be interconnected to perform one or 105. i n response to the commands a signal may be switched 

more operations on the various information streams. The through several islands (e.g., 406 and 300) prior to being 

studio output may be delivered to end-users (e.g., the public) stored in a storage unit (not shown) at the editing island (e.g., 

via ATSC broadcast, cable, telephone and satellite transmis- 30 407). 

sion and the like. The studio output may also be stored for An alternate mode of studio operation is to controllably 

later use in, e.g., a server or on CD-ROM or video tape. The operate one or more islands in a non-seamless mode. The 

invention is also useful in video teleconferencing and other non-seamless mode may be required in several circum- 

applications. stances where a splice or other transition between bitstreams 

While the streams delivered to broadcast customers must 35 must occur rapidly, and a range of bitstream degradation is 

meet, e.g., ATSC standards, it is not necessary to deliver all permissible. It must be noted that non-seamless switching 

of the internal studio information. For example, high bit -rate may produce errors which are propagated to subsequent 

studio formats are useful only within the studio or studio- islands receiving a degraded bitstream. These errors may be 

like environments. When dealing with splicing, there may be mitigated, if necessary, by, e.g., dropping damaged or infe- 

information within the stream that is meaningless to con- 40 rior access units or groups of access units (e.g., video 

sumer decoders, but which is necessary to studio splicing. frames) or by adding additional access units. For example, 

FIG. 4 depicts a block diagram of digital studio compris- if a to-stream having a short delay-parameter is to be spliced 
ing a plurality of interoperable islands and including the onto a from-stream with a long delay-parameter, the splicing 
invention. The digital studio 400 of FIG. 4 includes interop- operation is unlikely to be seamless (i.e., the buffer will 
erable islands 401, 402 and 404-409. The digital studio 400 45 likely overflow). In this case, frames may be dropped to 
also includes a first compressed bitstream stream source 110, avoid the overflow condition. Also, when a to-stream with a 
a second compressed bitstream stream source 120, a splicer long delay-parameter is to be spliced onto a from-stream 
300, a controller 105 and an optional splice monitoring unit. with a short delay-parameter, the splicer needs to adjust time 
The first compressed bitstream stream source 110, illustra- stamps to cause a number of frame repeats (i.e., add frames) 
tively a "live feed" from a transport stream encoder, pro- 50 while the buffer fills. The buffer may also be increased by 
duces a first MPEG-compliant transport stream S6. The splicing short, all-black frames on the end of a short delay- 
second compressed bitstream stream source 120, illustra- parameter sequence to build up the value of the delay- 
lively a server (e.g., a video disk, tape machine, or other parameter in current use. 

storage device) which stores video and audio elementary In the exemplary embodiments of FIGS. 1, 3 and 4, 

streams and transport encodes the stores streams to produce 55 splicing operations take place in operational units (e.g., 

a second MPEG -compliant transport stream S7. The first and splicing islands), such as routing switchers, play-to-air 

second compressed bitstream sources 110, 120 operate in switchers, production switchers or other switchers, 

substantially the same manner as previously described with Therefore, it is desirable to support a plurality of data 

respect to the bitstream splicing system 100 of FIG. 1. The formats and bitrates. For example, the so-called 422@HIGH 

digital studio 400 includes a controller 105 which performs 50 and 420@HIGH television studio formats each support 

those functions previously described with respect to the multiple picture formats and bit rates. Therefore, it may be 

bitstream splicing system 100 of FIG. 1, and other functions necessary to splice, e.g., a bitstream comprising a 1280 by 

which will be described below. Island 300 roughly equates 960 picture element, 60 Hz Progressive Scan picture onto 

to the splicer 300 of the bitstream splicing system 100 of the end of a bitstream comprising a 1920 by 1080 picture 

FIG. 1. 65 element, 59.94 Hz interlaced picture. Moreover, it may also 

Each of the islands receives a plurality of information be necessary to splice a 45 Mb/s stream onto the end of a 155 

streams. For example, islands 401 and 402 each receive Mb/s stream. 
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Both of the above example splices may be seamlessly and PES levels. In addition to the insertion of in-point and 

made if the streams being spliced have matching delay out-point markers, the delay-parameter associated with the 
parameters. Therefore, it is important that the controller that stream or splicing segment and an audio offset (i.e., a 
makes the splice decision know the delay parameters of the displacement of audio-frame boundaries from associated 

various streams to be spliced. The delay parameter of a 5 video frames) should also be inserted into one or more layers 

stream may be calculated by an operational unit receiving a of the information stream. The MPEG count-down feature 
stream or included within the stream as part of the stream's should ^ be ^ l0 indicate tnatj e „ ^ ou t-point is 

header information In the splicer 300 of FIG 3 switch approaching (decreasing^ positive countdown) or an 

comroUer ^340 includes a bitstream .calculator which calcu- . ^ has been transmitted (increa singly negative- 
ly L 7 J^^^^ in P ut t s,rcam * S * S7 ' Is 10 countdown). For various business reasons it may be desir- 

should be noted that the delay parameter calculation may . m ' M tlMkM mnrUorc nr -„ tn ' „ w DT7< - 

also be performed by the bitstream examiners 310A, 310B abl ^r ***** P "° r ? transmittm 8 « *J? E f 

. I-./* or ATSC signal to an end-user (i.e., consumer). While 

or the optional splice monitor 130. , .°7. ,. \. .... ' . . 

, .. , end -users might want to splice the video, it is important that 

Another critical aspect of splicing information streams is lfa M bc able tQ cli QUt conimercials automatically, 

the determination of in-point and out-pouit locations in the ' , 

streams to be spliced. To properly perform a seamless splice 15 ^ above-described redundancy of marking provides a 

it is necessary to find the in-point of me to-stream and the maximum flexibility to system designers and provides a 

out-point of the from-stream. Moreover, a splice segment redundancy of operation which helps to ensure that splicing 

may include m- P oints and out-points having different delay- °P eratlons » fact seamlessly made (i.e., made at 

parameter values. There are several options available for 2Q appropriate m-points and out-points), 

finding the appropriate splice points. L Bitstream Generation 

First, the entire to-stream or from-stream may be analyzed 

by the splicer in real time (i.e., "on the fly"). A real-time To hel P eQSUre seamless splicing it may be necessary to 

analysis is difficult for a to-stream because an in-point create the bitstreams to be spliced m a certam manner. There 

cannot readily be deduced from the stream without playing 25 are tw0 facets to tne creation of bitstreams that can be 

the stream to its end. Even if it were known that the stream spliced; the creation of the stream content, and the insertion 

is well-behaved, the length of an I-frame is not known in of appropriate splice control information (i.e., in-point and 

advance. By the time the first I-frame has ended, and its out-point markers). For simplicity, assume that a desired 

length is known, it is probably too late for the information value of the delay-parameter is known in advance. In 

to be used. It must be noted that this problem may be 30 addition, other goals, such as how often an in-point is 

overcome by using, e.g., a more powerful computing device. wanted, are also known. 

A real-time analysis is easier for a from-stream because the For the simplest case, encoding all I-frame low-delay 

delay-parameter of the from-stream is known (from the format, the creation of the bitstream becomes a matter of 

in-point or otherwise), the presentation time-stamps in the rate-control. For each frame, there is a not-to-be-exceeded 

stream indicate when frames leave the decode buffer, and 35 bit-count. The rate-control task is to encode each frame with 

bit-counts (or packet counts) indicate when the frames enter the best quality possible within the bit-budget. The per- 

the decode buffer. The frame rate is also known from frame bit budget is computed as the transmission-bit-rate 

sequence headers. With the foregoing information it divided by the frame-rate. 

becomes possible to calculate, at the start of a new frame, For complex GOP encoding a forward analysis of the 

that the preceding video has left the buffer in an out-point 40 created stream may be made. The allocation of bits among 

status. This information arrives only a little bit late. frames must be done to assure that a decoder buffer doesn't 

Second, an external table may be created to contain underflow. A first constraint which may be applied to the 

indications of where splice points are. This approach created stream is the defining of a splice segment as a fixed 

assumes that the information about in-point and out-point GOP structure (e.g., 13 frames arranged in the following 

locations has been computed elsewhere (e.g., during a 45 display order: ". . . IBBPBBPBBPBBP ... "). This approach 

stream encoding process). This approach requires that the is straightforward a the expense of unnecessarily degraded 

in-points and out-points be indexed in some manner (e.g., picture quality. For example, a scene cut on the last P frame 

Nth packet from a marker, first packet after a time-of-day of an . . IBBPBBPBBPBBP ..." GOP would be 

reference, and the like). This approach also requires the reproduced with a very small bit budget. Unfortunately, 

updating of a splice table associated with an information 50 there is no single GOP structure that is ideal for all appli- 

stream as the information stream is processed, and the cations. Moreover, the loss of flexibility implied in this 

transmission or regeneration of the table as the information approach is probably unacceptable, 

stream is transmitted (e.g., via satellite link). It should be A second constraint which may be applied to complex 

noted that the use of separate information tables to deter- GOP encoding is the insertion of in-points at out-points at 

mine splice locations is practical within a server or island of 55 predetermined time intervals (e.g., 2 and 0.5 seconds, 

operability, though this use of tables may be less practical respectively). This approach does not require the use of a 

when transmission encoding and decoding are to be per- specific GOP structure, therefore the encoder is free to select 

formed on the stream. frame type based upon the input pictures. 

Third, in-point and out-point markers may be placed There are various rate-control issues to be resolved when 

within the information stream direcdy. An MPEG compliant 60 switching between MPEG streams or splice segments. As 

information stream includes header portions where such a such, it may be necessary to constrain the contents or size of, 

marker may be included. There are header portions suitable e.g., coded picture bitstreams. One rate-control issue 

for in-point and out-point marker insertion at the system involves the amount of data transmitted to a decoder buffer, 

level, transport level and PES level. There are also oppor- For example, the decoder buffer will not overflow if the 

tunitics to insert markers in the elementary streams. 65 buffer contents (measured in bits) at any out-point is less- 

Both in-points and out-point should be marked and, than-or-equal-to the decoder buffer contents (measured in 

ideally, the marking should occur at the system, transport bits) measured at the most recent in-point. It is not necessary 
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to know the actual number of bits, it is only necessary to combined in a logical manner to determine a splice time and 

ensure that the number of bits in the decoder buffer does not select appropriate streams for splicing, 

increase from in-point to out-point. It must be noted that After a decision is made it is communicated to the 

so-called "stuffing bits" are not counted, since these bits are appropriate operational unit. Since there is a possibility of 

only included to meet specified transmission rates and 5 errors (e.g., due to the complexity of combining multiple 

disappear from the buffer when the real bits are used (i.e., logical events) it is important for the controlling entity to 

stuffing bits do not accumulate in the buffer). monitor the splicing process closely and adapt to error 

Another rate control issue involves the presentation time conditions and other conditions, 

of decoded information frames in relation to the time the As a result of the decision, and part of making the splice 

buffer receives the next frame. For example, the decoder 10 W ork, is the process of assuring that the correct bitstream is 

buffer will not underflow if, at an out-point, the time applied to the correct input of the operational unit at the 

(measured in time units) to display the not-yet displayed correct time. This is a function of overall system operation, 

buffer contents is greater than the time (measured in time an d ma y involve synchronization of information flow from 

units) for the first I-frame to be delivered to the decoder at a server to the switcher. After receiving the splicing decision 

the specified bit rate. In other words, the next 1-frame (the IS the splicer will make the splice. The process of seamless 

first frame of the to-stream) must be delivered to the decoder splicing requires that the to-stream is entered at an in-point, 

buffer before the buffer is emptied. the from-stream is exited at an out-point and, possibly, the 

The above-described amount of time may be defined at management of one or more buffer and timing parameters, 
the "Delay-Parameter" for the stream. The frame sizes 

(measured in time to transmit the frames at the operating bit 20 Illustrative Examples of Video Splicing 
rate) must be consistent with the operating delay parameters above-described concepts will now be illustrated 
to ensure seamless splicing. The delay parameters are the using ^vatd examples of MPEG bitstream splicing opera- 
end-to-end VBV size (measured in time) and the VBV tions fifSt exampIe fc an all I-frame low delay splicing 
contents (measured m time) at the beginning/end of a example. A to-stream comprises either 24 or 30 frames per 
stream. An additional, globally defined value is the maxi- second (fps) video streams including only I-frames. The 
mum size of physical buffers (in bits). This maximum size delay parar n ete r of the to-stream is equal to one frame time 
must be greater than the maximum VBV size implied by the at the slowest frame rate (i e >f 42 mS if 24 fps). In this 
MPEG profile and level indication criteria. Finally, the example each of the I-frames contains fewer bits than can be 
decode time stamp at an out-point of a from-stream should ^ nt m one unit 0 f display time (i.e., one 42 mS frame time) 
be one frame time of the stream greater that the DTS and at the bit rate for the frame If me bit rate ^ 150 w ^ a 30 
PTS of the last frame of the from-stream. fp S framc contains no more than 5 Mb. If the bit rate is 150 

As previously mentioned, it is important to distinguish Mb/s, a 24 fps frame contains no more than 6.25 Mb. 

between a splicing decision and the actual process of splic- When t he last bit of a from-stream having the same 

ing. A splicing decision is made by some human. The 35 parameters has entered a decoder buffer, the presentation 

decision may be made in the process of generating a list of lime ^amp indicating when the last frame is to be presented 

programming to be transmitted by a television studio or in has a vauie 42 m S in the future. -Thus, if the from-stream 

real time as the studio is transmitting. The splicing decision were at 30 Hz (33 mS frame rate ) f 9 ms> af ter lhe ou t-point 

may be made by some surrogate process, such as a prepro- tDe last frame 0 f the from stream will be taken from the 

grammed command to splice a station identification 4Q decoder buffer, and 33 mS later the first frame of the 

announcement into the studio transmission every day at to-stream will be needed. If the to-stream is also 30 Hz, the 

12:05 AM. The decision may be to splice at a particular time first f rame will have been delivered 9 mS before it is needed, 

in the future or immediately. If mc from-stream were at 24 Hz, and the to-stream were 

Several parameters of the streams to be spliced may be also at 24 Hz, the to-stream frame arrives just in time. If the 

known at the time of the decision, though these parameters 45 from-stream were at 60 Hz, (17 mS) then when the last bit 

may change prior to the actual splicing operation. For of the from-stream is delivered, the decoder buffer contains 

example, the splicing decision is usually made with some 2 frames (33 mS) and the decoder will not use the first of 

knowledge of the to-stream, such as the stream length, VBV them for 9 mS. If the bit -rates of the from and to sequences 

delay parameter and the like. It is possible for the from- differ, as long as the bits are delivered at the rate corre- 

stream to be unknown at the time of the decision (e.g., the 50 sponding to the sequence bit-rate, the time to load frames 

daily message is inserted into whichever stream is being remains correct. To summarize, streams are coded with a 

transmitted at the time of insertion). bit-count between the in-point and following out-points that 

A decision contains the following elements. First, the k calculated from the bit-rate and the frame-time (i.e., 

operational unit which sources the to-stream, the operational bit-rate * frame-time). The presentation time-stamps are set 

unit which performs the splice and the stream or segments 55 to values that all agree with the delay-parameter (Le., first 

to be spliced. Second, the time the splice is to take place. The frame presented delay-parameter after the first bit arrives.), 

time may be "now," a particular time of day or the occur- The second example is a complex GOP transmission 

rence of some logical condition. "Now" means make the format. For purposes of the second example, it is assumed 

next splice after the arrival of the splice-now message. The that stream is a 30 frame per second video (and associated 

now decision may arise from direct human action (e.g., 60 audio) stream having delay-parameter of 250 mS, a display 

button-press) or some external controlling process deciding order of . . IBBPBBPBBPBBPBBP ..." and transmission 

to send a splice-now message. The logical condition may be order of . . IPBBPBBPBBPBBPBB ..." (where "I" 

the occurrence of a time-code (e.g., SMPTE) in a particular represents an I-frame, "P" represents a P-frame and "B" 

information stream, a time stamp (e.g., PTS or DTS) in a represents a B-frame).This GOP structure includes in-points 

video or audio stream, a reference time (e.g., PCR) in an 65 on the I-frames and out-points on the frame immediately 

information stream or some other detectable event (e.g., an preceding the I-frames. At each out-point, the PTS associ- 

input stream PID changes). The logical events may be atcd with the last P frame is 250 mS in the future. The rate 
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control ensures that the decoder buffer doesn't underflow on respect to each other. A Presentation Time Stamp (PTS) 

the I-frame. It must contain less than 250 mS worth of bits. exists in each audio stream. The audio and video PTSs refer 

The valid-MPEG constraint implies that the P-frame fol- to the same reference to allow the required synchronization. 

lowing the I frame also doesn't underflow. It is not necessary When the splice is complete, the to-stream becomes the 

that the I-frame use all 250 mS. If the I-frame does use all 5 output stream. It is important to note that, particularly due to 

250 ms, the next P-frame must use less than 33 mS (i.e., if audio constraints, the process of switching may extend in 

the I-frame used 230 ms, the next P-frame would be time before and after the actual switch instant. 

permitted, though not required, to use 53 mS). Audio information frames in an information stream are 

The third example is a multiple output example. For ideally located within a limited time difference from respec- 

purposes of the third example it is assumed that stream is a 10 tive video information arrival at the end of the decoder 

30 frame per second video (and associated audio) stream buffer. If there is a Vi second end-to-end video buffer delay, 

having the following GOP structure: . . IPPPPPPPPPPP- then audio packets should be approximately Yi second later 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPIPPPP ..." The in a transmission stream than corresponding (i.e., having the 

stream is also assumed to have a delay-parameter of 250 mS same presentation time stamps) video packets. If this 

and a transmission bitrate of 20 Mb/s (i.e., 670 Kb per 15 assumption is correct, then the switching operational unit 

frame). If an I-frame takes 231 mS and P-frame takes 20 mS must save audio information from the from stream for this 

then, after 15 frames, the decoder buffer contents have Vi second after the video switch. The source stream must 

subsided to a level below the level at the in-point to the continue for Vi second after the splicer has switched to 

stream. This may be calculated using an equation such as the another stream. It is also amusing to contemplate rapid 

following: 20 switching among several streams. 

Three approaches to splicing audio will now be described. 

(23i+20N)=33-(N + i) the overlapped audio packets raay simply ^ broken. 

The above equation represents the time for bits entering, This is not the most desirable approach because it relies 

starting with an I-frame's 231 mS worth of data, followed by u P<>n the CRC to prevent the use of partial packets. If the 

N 20 mS P-frames, being equated to the time for the same 25 CRC fails one time io 64 K packets, at about 30 packets/ 

N+l frames-worth of bits to be taken from the buffer (N+l) second, every few thousand seconds there is a potential 

frames, each taking 33 mS. In this example each I-frame undetected error. When a broken-packet CRC fails, it fails 

may be an in-point and all the P-frames after the fifteenth cver y time lne packet is used. This means that a failure that 

P-frame may be out-points. produces a click may end up reproduced every time the same 

The decoder buffer reacts as follows. At the splice point, 30 splice is made. This requires the use of a garbage-collecting 

the buffer contains 7 frames and a presentation time stamp process to remove broken audio frames. 

indicates that the last frame is to be presented in 250 mS. A second approach to splicing audio is to include "unfin- 

The 7 frames each use 20/33 of the bits/frame quota (i.e., ished" from-stream audio frames (i.e., those overlapping a 

each P-frame frame is 400 Kb and the buffer contains 2.8 video splice) in the resultant stream. The to-stream audio 

Mb). Over the next 7 frames, the I-frame comes in, increas- frames are then retimed such that they butt up against the 

ing the buffer contents. Since each I-frame adds 670 Kb and "unfinished" from-stream audio frame. This technique 

each P-frame taken out removes 400 Kb the buffer contains assures continuous audio at the expense of continuous 

4.7 Mb after the seven P-frames are taken out. The I -frame inspection, buffering and adjustment of audio frames and 

is then presented, removing 4.6 Mb from the buffer and, ^ packets. The first complete to-stream audio frame is selected 

therefore, leaving 100 Kb in the buffer. The delay in the as the first audio frame to play because the to-stream frame 

buffer is approximately zero. Each P-frame now adds 400 which has already started is likely to be delayed too much to 

Kb in 20 mS and every 33 ms 400 Kb is used. Thus, delay match the end of the "unfinished" from-frame. This tecb- 

in buffer increases by 13 mS every frame time. After 15 nique also causes a slightly distorted lip-sync. 

frames, the delay stored in the buffer has reached the 4J A third approach to splicing audio is to maintain align- 

delay-parameter value. At this time a splice to another mC nt of audio with its corresponding video, that is, to leave 

sequence may be made because the buffer is able to receive a gap between the end of the from-stream audio frame and 

an I-frame. the beginning of the to-stream audio frame. This approach 

^ . . «i i advantageously relies on the MPEG decoder specification 

Compressed Audio Splicing ^ which rcquircs that audio framc gaps arc mu{cd [n ^ 

The following discussion of splicing of compressed audio approach the audio presentation-time-stamps are adjusted by 

is limited to the issue of splicing combined video-audio the same adjustment amount applied to the video frames. In 

streams in the audio -follows-video mode. The composition this manner lip sync is maintained. The third approach is 

of streams from separately edited audio and video streams is especially useful when many splices (e.g., the creation of a 

not considered here. 55 sequence from a succession of short splice segments) may 

Compressed audio is carried in frames. Each audio frame produce audio artifacts due to the muring, 

is of fixed duration and contains a fixed number of bits. FIG. 5 depicts a plurality of splicing scenarios involving 

Unfortunately, the audio frame size, or duration, is different audio alignment which illustrate aspects of audio-video 

from any of the video frame sizes, or durations. This means splicing, assuming the above cited third approach is used to 

that audio frames will not align with splice points. Audio go maintain alignment of audio with its corresponding video, 

frames can be considered to be randomly aligned with the FIG. 5A depicts the simple splicing case where both audio 

video. Therefore, when making a splice, the alignment of the streams align with their corresponding video. The splicer 

audio with the video will be different for the to-stream and delays both to-streams and simply switches at the splice 

the from-stream. point. 

It is desirable to ensure alignment of audio information to 65 FIG. 5B depicts the splicing case where the from-stream 

corresponding video information (i.e., "hp-sync"). That is, video and from-stream audio arc aligned, but the tn-stream 

the audio and video must remain properly phased with video and to-stream audio are not aligned. When the splice 
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is made, a partial to-slream audio frame is discarded. The 
next complete to-audio frame is passed to the output with 
appropriate delay. 

FIG. 5C depicts the typical splicing case where both audio 
streams do not align with the corresponding video streams, s 
It can be seen that a from-stream audio frame has already 
begun before the splice point. This audio-frame is buffered 
aod transferred to the output. It doesn't end until some 
fraction of a frame time after the splice. The to-stream audio 
frame that spans the splice-point cannot be used. The next io 
to-stream audio frame also cannot be used. It begins too 
early, and would overlap the last from-stream audio frame. 
The first to-stream audio frame that appears in the output 
stream begins D time units after the splice-point. This delay 
may be as much as two audio frames. 15 

In the above example (FIG. 5C) the lip-sync is preserved, 
but as much as 32 mS of from-stream audio overlaps the 
to-stream video. Also, the first to-stream audio begins as late 
as 64 mS after to-stream video begins. Finally, the splicer 
performing the splicing operation must buffer a whole audio 20 
frame in each work buffer. 

Ideally, every audio frame includes a PTS. It is possible 
that some equipment manufacturers only include a PTS 
every, e.g., third audio frame. In this case, or the case where 
there is no audio PTS, a splicing operation may be per- 25 
formed after calculating a virtual time-stamp. The virtual 
time stamp is derived from the approximate real-time delay 
of audio-frames from video reference time-stamps. The 
virtual audio-time-stamp is then incremented by the (known) 
audio-frame duration on successive audio-frame starts. This 30 
process may be implemented as a backup process to ensure 
that non-time stamped audio streams to not get into a studio 
where time stamps are crucial to the operation of the studio. 

Auxiliary Data Splicing 

Auxiliary data is included in many MPEG streams. This 
data is usually present as contiguous, unbreakable streams of 
unknown length. By contrast, a compressed audio stream is 
relatively well-behaved and predictable. The auxiliary data 
stream may be associated with a corresponding video stream 40 
having a delay that is not now specified. 

There are several methods for handling auxiliary data in 
a splicing operation, including: 1) ignore it and splice the 
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What is claimed is: 

1. A method for splicing bitstreams, said bitstreams com- 
prising a plurality of information segments representative of 
a sequence of information frames, said method comprising 
the steps of: 

monitoring a first bitstream to detect an exit indicium, said 
first bitstream being coupled to an output and including 
at least one of said exit indicium, said exit indicium 
indicative of an appropriate last information segment of 
a splicing segment; 

providing a second bitstream, said second bitstream 
including at least one entrance indicium, said entrance 
indicium indicative of an appropriate first information 
segment of a splicing segment; and 

coupling said second bitstream to said output in response 
to a splice decision signal, a detection of said exit 
indicium in said first bitstream and a detection of said 
entrance indicium in said second bitstream, said step of 
coupling comprising the steps of: 
decoupling said first bitstream from said output at a 

detected exit indicium; 
coupling said second bitstream to said output at a 

detected entrance indicium; and 
transmitting a splice acknowledgment signal. 

2. The method of claim 1, wherein: 

said splice decision signal comprises at least one of a 
desired temporal parameter, a desired splice quality 
parameter and a priority parameter; and 

said splice acknowledgment signal comprises at least one 
of an actual temporal parameter and an actual splice 
quality parameter. 

3. The method of claim 2, wherein: 

said desired temporal parameter is indicative of a desired 
time to perform said coupling step; 

said desired splice quality parameter is indicative of a 
minimum splice quality level, said minimum splice 
quality level being less than or equal to a seamless 
splice quality level; 

said actual temporal parameter is indicative of a time said 
coupling step was performed; and 

said actual splice quality parameter is indicative of a 
quality level of a bitstream produced during said cou- 
pling step. 

4. A method for splicing bitstreams, said bitstreams corn- 



auxiliary data at the same time as the video data; 2) insert 

auxiliary data through a separate path to, e.g., a play-to-air 45 prising a plurality of information segments representative of 

switcher (this data may comprise a program guide or other a sequence of information frames, said method comprising 

consumer-related information); and 3) define a set of seg- tne steps of: 



mentation markers for auxiliary data and rely upon these 
markers in switchers to keep segmentation correct (this 
requires knowledge about the content of auxiliary data 50 
streams and the lengths of auxiliary data segments within 
those streams). The auxiliary data may also be switched with 
or without a delay, and the delay may be a parameter passed 
to the switcher by a decision making controller. The auxil- 
iary data may be input to the splicer via an auxiliary input. 55 
Input arriving on the auxiliary-data input may be buffered 
and inserted into the output stream on a space-available 
basis as a replacement for null packets. In this case it 
becomes some other system unit's responsibility to align 
such data within streams, and to provide channel capacity 60 
for inserted aux-data by, e.g., reducing a video stream data 
rate. 

Although various embodiments which incorporate the 
teachings of the present invention have been shown and 
described in detail herein, those skilled in the art can readily 65 
devise many other varied embodiments that still incorporate 
these teachings. 



monitoring a first bitstream to detect an exit indicium, said 
first bitstream being coupled to an output and including 
at least one of said exit indicium, said exit indicium 
indicative of an appropriate last information segment of 
a splicing segment; 
providing a second bitstream, said second bitstream 
including at least one entrance indicium, said entrance 
indicium indicative of an appropriate first information 
segment of a splicing segment; and 
coupling said second bitstream to said output in response 
to a control signal, a detection of said exit indicium in 
said first bitstream, a detection of said entrance indi- 
cium in said second bitstream, said step of coupling 
comprises the steps of: 

comparing a delay parameter associated with said first 

bitstream to a delay parameter associated with said 

second bitstream; and 
coupling said second bitstream to said output in 

response to said delay parameters being substantially 

equal. 
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5. The method of claim 4 wherein said control signal 
comprises a splice decision signal, and said step of coupling 
further comprises the steps of: 

decoupling said first bitstream from said output at a 

detected exit indicium; 
coupling said second bitstream to said output at a detected 

entrance indicium; and 
transmitting a splice acknowledgment signal. 

6. The method of claim 5 further comprising the step of: 
storing, in response to a detected entrance indicium in said 

second bitstream, said appropriate first information 
segment and a plurality of successive information seg- 
ments associated with said detected entrance indicium; 
and wherein: 

said splice decision signal includes at least a priority 
parameter, and said step of coupling further comprises 
the steps of: 

increasing or decreasing a delay parameter associated 
with said stored information stream in response to a 
predetermined priority parameter. 

7. A method for splicing a first information stream into a 
second information stream to produce an output information 
stream, each of said information streams comprising a 
plurality of information segments representative of a 
sequence of information frames, said method comprising the 
steps of: 

coupling said second information stream to an output to 
produce said output information stream; 

detecting an intra-coded information frame in said first 
information stream, said first information stream 
including at least one of said intra-coded information 
frame, said intra-coded information frame indicative of 
an appropriate first information segment of a splicing 
segment; 

storing, in response to said intra-coded information frame, 
said first information segment in a first portion of a 
memory, said information segments representative of a 
sequence of information frames succeeding said intra- 
coded information frame being stored in a second 
portion of memory, and a plurality of successive infor- 
mation segments, said step of storing comprises the 
steps of: 

storing information segments representative of said 
intra-coded information frame in a first portion of a 
memory; 

storing information segments representative of a 
sequence of information frames succeeding said 
intra-coded information frame in a second portion of 
memory; 

repeating said step of storing each time an intra-coded 
information frame is delected in said first informa- 
tion stream; 

detecting an exit indicium in said second information 
stream, said second information stream including at 
least one of said exit indicium, said exit indicium 
indicative of an appropriate last information segment 
of a splicing segment; 

retrieving, in response to said exit indicium, said stored 
information segments; and 

coupling said retrieved information segments to said 
output to produce said output information stream. 

8. Apparatus for splicing information streams, said appa- 
ratus comprising: 

a buffer receiving a first information stream and produc- 
ing a buffered information stream; 

a bitstream examiner, coupled to said buffer, receiving 
said first information stream and responsively causing 65 
said buffer to position an entrance point of said buffered 
information stream at an output of said buffer; 
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a switch, for coupling either said buffered information 
stream or a second information stream to an output; and 

a switch controller, for transmitting a splice acknowledge- 
ment signal, coupled to said switch and receiving said 
second information stream and a splice decision signal, 
for monitoring said second information stream and, in 
response to said splice decision signal and the detection 
of an exit point in said second information stream, 
causing said switch to couple said buffered information 
stream to an output. 

9. The apparatus of claim 8, wherein: 

said switch controller compares a delay parameter asso- 
ciated with said buffered information stream to a delay 
parameter associated said second information stream; 
and 

said switch controller, in response to said delay param- 
eters being substantially equal, causes said switch to 
couple said buffered information stream to said output. 

10. The apparatus of claim 9, wherein said switch control 
signal comprises a splice decision signal including at least a 
priority parameter, and said switch controller traosmits a 
splice acknowledgment signal. 

11. The apparatus of claim 9, wherein said switch con- 
troller is coupled to said bitstream examiner, and wherein: 

said switch controller, in response to a predetermined 
priority parameter of said switch control signal, causes 
information to be added or deleted from said buffer 
such that said delay parameter of said buffered infor- 
mation stream is, respectively, increased or decreased. 

12. Apparatus for splicing information streams, said appa- 
ratus comprising: 

a buffer receiving a first information stream and produc- 
ing a buffered information stream; 

a bitstream examiner, coupled to said buffer, receiving 
said first information stream and responsively causing 
said buffer to position an entrance point of said buffered 
information stream at an output of said buffer, said 
buffer being flushed by said bitstream examiner each 
time an entrance point is detected in said first informa- 
tion stream; 

a switch, for coupling either said buffered information 
stream or a second information stream to an output; and 

a switch controller, for transmitting a splice acknowledge- 
ment signal, said splice acknowledgement signal com- 
prising at least one of an actual temporal parameter and 
an actual splice quality parameter, coupled to said 
switch and receiving said second information stream 
and a splice decision signal, for monitoring said second 
information stream and comprising at least one of a 
desired temporal parameter, a desired splice quality 
parameter and a priority parameter, and the detection of 
an exit point in said second information stream, causing 
said switch to couple said buffered information stream 
to an output. 

13. The apparatus of claim 12, wherein: 

said desired temporal parameter is indicative of a desired 
time to couple said buffered information stream to said 
output; 

said desired splice quality parameter is indicative of a 
minimum splice quality level, said minimum splice 
quality level being less than or equal to a seamless 
splice quality level; 

said actual temporal parameter indicative of a time said 
buffered information stream was coupled to said out- 
put; and 

said actual splice quality parameter is indicative of a 
quality level said output information stream. 
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